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REMARKS 

Claims 1-11 were pending in the application. Claims 1-11 were rejected. Claims 
1-1 1 are amended. Claim 1 is the independent claim. Reconsideration of the amended 
application is respectfully requested. 

The examiner required amendment of the claims to bring the claim language in 
conformance with provisions of the MPEP. The claims are amended, and it is 
respectfully submitted that the amended claim language complies in all respects with the 
MPEP. 

The examiner required the addition of an abstract of the disclosure. An abstract 
has been added on a separate sheet. 

The examiner required amendment of the specification in conformance with 
content and heading provisions of MPEP 608.01(a). The written description has been 
amended to amend and add headings as applicable. 

The examiner rejected claims 1-11 under 35 USC §101 as being directed to non- 
statutory subject matter. The claims are amended so as to make it clear that they are not 
directed merely to abstract ideas, but rather recite a process that results in a practical 
application producing a tangible result to form the basis of statutory subject matter under 
35 USC § 101 . That is, the claims recite a method for training a neural network. The 
utility of neural networks is well known, and the method for training the neural network 
recited in the claims results in at least one practical application, namely, simplification of 
the structure of the neural network. The advantages of such simplification are noted in 
the Background section of the present invention, as well as on page 116 of the reference 
cited by the examiner. The rejection, therefore, should be withdrawn. 
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The examiner rejected claims 1-1 1 under 35 USC §112, first paragraph, as not 
reciting how to practice the invention. Claims 1-1 1 are amended to recite the invention 
such that one of ordinary skill in the art can practice the invention. That is, the claimed 
method for training a neural network is recited as a series of actions performed on a 
recited structure, resulting in a tangible, practical application, such as that included at the 
end of claim 1 . All of the actions are described such that one of ordinary skill in the art 
can carry them out, as is the structure on which the actions are preformed. The rejection, 
therefore, should be withdrawn. 

The examiner rejected claims 1-11 under 35 USC §112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
that the applicants regard as the invention. The examiner noted particular language that 
he considers to be indefinite. The claims are amended for improved definiteness, and it is 
respectfully submitted that the amended claims satisfy the requirements of 35 USC §112, 
second paragraph. The rejection, therefore, should be withdrawn. 

The examiner rejected claims 1-11 under 35 USC § 102(b) as being anticipated by 
Mehrotra et al. 

Independent claim 1 recites a method for training a neural network in order to 
optimize the structure of the neural network. The neural network includes an input layer 
having a plurality of input neurons that receive the input data, at least one intermediate 
layer having a plurality of intermediate neurons, an output layer having a plurality of 
output neurons that provide output signals, and a multiplicity of synapses, wherein each 
said synapse interconnects two neurons of different layers, defining a sending direction 
fi-om the input layer to the output layer. The training method includes identifying and 
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eliminating synapses that have no significant influence on the curve of the risk function. 
First and second sending neurons are selected that are connected to the same receiving 
neuron by respective first and second synapses. It is assumed that there is a correlation of 
response signals fi-om the first and second sending neurons to the same receiving neuron. 
The first synapse is interrupted and a weight of the second synapse is adapted in its place. 
The output signals of the changed neural network are compared with the output signals of 
the imchanged neural network. If the comparison result does not exceed a predetermined 
level, the first synapse is eliminated, thereby simplifying the structure of the neural 
network. 

Mehrotra et al. disclose the structure of a neural network in Fig. 1.14, as noted by 
the examiner. On page 1 16, Mehrotra et al. also disclose "pruning" the network, that is, 
eliminating nodes and connections that are considered unimportant. In Fig. 4.6, Mehrotra 
et al. describe a generic pruning algorithm, by which nodes or connections are deleted is 
the removal does not penalize performance beyond a desirable tolerance level. Mehrotra 
et al. then outline, on pages 116-118, four procedures that can be used to implement the 
network pruning methodology. None of these procedures includes selecting first and 
second sending neurons that are connected to the same receiving neiuron by respective 
first and second synapses, assuming a correlation of response signals from said first and 
second sending neurons to the same receiving neuron, interrupting the first synapse and 
adapting in its place a weight of the second synapse, comparing the output signals of the 
changed nexiral network with the output signals of the unchanged neural network, and if 
the comparison result does not exceed a predetermined level, eliminating the first 
synapse, as recited in claim 1. 
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The first procedure calls for eliminating connections having weights of small 
magnitude, as well as eliminating their associated nodes. Presumably, this procedure 
involves comparing synapse weights against a threshold, and eliminating synapses having 
weights below the threshold. Mehrotra et al., in describing this procedure, do not 
disclose selecting two sending neurons, assuming a correlation, interrupting a synapse 
and adapting a weight of another synapse, comparing reactions of the network, and then 
eliminating a synapse, as required by claim 1. In contrast, this procedure merely checks 
the weights of synapses in making a determination of whether to eliminate the synapse. 
Thus, this procedure does not anticipate the invention recited in claim 1. 

The second procedure calls for determining the effect that a change in the synapse 
weight has on network output, and eliminating synapses if the change is acceptable. The 
change in synapse weight is not disclosed to be based on a weight of another synapse 
having a different sending neuron and the same receiving neuron, as required by claim 1. 
Further, no pair of neurons is selected for weight adaptation and no correlation 
assumption is made, also as required by claim 1 . Thus, this procedure does not anticipate 
the invention recited in claim 1. 

The third procedure calls for pruning input nodes, and does not address 
identifying and eliminating synapses. Thus, this procedure does not anticipate the 
invention recited in claim 1. 

The fourth procedure calls for identifying weights to be pruned from a network by 
examining the second derivatives of the error function contained in the Hessian matrix, 
using a formula to approximate the change in output resulting firom a small perturbation 
in a selected weight. Pairs of neurons are not selected, weights are not substituted, and 
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actual changes based on the substitution are not made, as required by claim 1 . Thus, this 
procedure does not anticipate the invention recited in claim 1. 

In summary, Mehrotra et al. describe four pruning procedures for simplifying a 
neural network. None of these procedures includes all of the elements of the method 
recited in claim 1. That is, none of the disclosed procedures mcludes the process of 
selecting first and second sending neurons that are connected to the same receiving 
neuron by respective first and second synapses, assuming a correlation of response 
signals fi-om said first and second sending neurons to the same receiving neuron, 
interrupting the first synapse and adapting in its place a weight of the second synapse, 
comparing the output signals of the changed neural network with the output signals of the 
unchanged neural network, and if the comparison result does not exceed a predetermined 
level, eliminating the first synapse, as recited in claim 1. The rejection of claim 1, 
therefore, should be withdravm. 

Claims 2-11 depend fi-om claim 1, and therefore also are not anticipated by 
Mehrotra et al., for the reasons noted with respect to claim 1, and in view of the 
additional features recited in the dependent claims. For example, claim 2 recites that the 
first and second selected sending neurons are located on the same layer. The examiner 
noted that Mehrotra et al. disclose sending neurons located on the same layer, in Fig. 
1.14. Fig. 1.14 does show that sending neurons exist on the same level, but not that two 
sending neurons selected as part of the process recited in claim 1 are located on the same 
layer, because that process is not disclosed by Mehrotra et al. For at least the reasons 
noted above, Mehrotra et al. also does not anticipate the invention recited in claims 2-11. 
The rejection of these claims, therefore, should be withdrawn. 
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Based on the foregoing, it is submitted that all requirements have been satisfied, 
and that all objections and rejections have been overcome. It is therefore requested that 
the Amendment be entered, the claims allowed, and the case passed to issue. 
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Respectfully submitted. 
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